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Abstract 

Let T be a stopping time associated with a sequence of independent random variables 
Z\, Z2, ... • By applying a suitable change in the probability measure we present relations be- 
tween the moment or probability generating functions of the stopping time T and the stopped 
sum St = Z\ + Z2 + ... + Zt- These relations imply that, when the distribution of St is known, 
then the distribution of T is also known and vice versa. Applications arc offered in order to 
illustrate the applicability of the main results, which also have independent interest. In the first 
one we consider a random walk with exponentially distributed up and down steps and derive the 
distribution of its first exit time from an interval (—a, b). In the second application we consider 
a series of samples from a manufacturing process and we let Zi,i > 1, denoting the number of 
non-conforming products in the i-th sample. We derive the joint distribution of the random 
vector (T,St), where T is the waiting time until the sampling level of the inspection changes 
based on a fc-run switching rule. Finally, we demonstrate how the joint distribution of (T, St) 
can be used for the estimation of the probability p of an item being defective, by employing an 
EM algorithm. 
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1 Introduction 



In several areas of applied science researchers are interested in studying the time T to take a 
given action, based on sequentially observed random variables (rv's) Z\, Z2, . . . , as well as in 
the associated partial sums S n = Z\ + Z2 + • • . + Z n , n = 1, 2, ... . The waiting time T and the 
corresponding random sum St are usually referred to as stopping time and stopped sum respectively. 
Stopping time problems arise in many diverse scientific areas such as sequential analysis, quality 
control, mathematical finance, operations research, biology, actuarial science, etc. For a gentle 
introduction to the theory of stopping times and stopped sums, the interested reader is referred 
to Karlin and Taylor (1975). For a more thorough investigation of the theory of stopped random 
walks we refer to Gut (2009). 

When studying the distribution of T in a sequence of independent and identically distributed 
(iid) trials, the stopped sum St also provides useful information about the nature of the statistical 
experiment. The pioneering work of Abraham Wald (1945) in the area of sequential analysis 
established powerful identities that relate the distributional properties of T and St- These identities 
are usually referred to as Wald's (fundamental) Identity and Wald's (first) equation and they are, 
respectively, given by 

E((M z (w))- T e wS n = h (1) 

where M z (w) = E(e wZ ), and 

E (St) = E(Z)E (T) . (2) 

In a recent article Antzoulakos and Boutsikas (2007) established a particular relation between 
the distributions of T and St- More specifically, they considered the waiting time T r until the 
r— th occurrence of a pattern £ in a sequence of binary trials Z\,Zi,... and the total number 
of successes St t observed until that time, and established a direct method to obtain the joint 
probability generating function (pgf ) of (T r , St t ) from the pgf of T r only. In this paper we extend 
the aforementioned result for any distribution of the Zj's and any stopping time T, determining 
the joint distribution of (T, St) from the distribution of T or St- 

The organization of the paper is as follows: In Section 2 we state the main identities that connect 
the distributions of T and St, along with the required theoretical backup. An important part of 
our work is comprised of the applications that are presented in Section 3. These applications, not 
only serve as an illustration of the applicability of the results of Section 2, but they also have an 
interest on their own. In the first one we consider the first exit time T from an interval (—a,b) 
(a > or a = 00) of a random walk Si, i = 1,2, with exponentially distributed up and down 
steps. By identifying the distribution of St we extract an exact formula for the pgf of the boundary 
crossing time T. In the second application we consider a sequence Zi, i = 1,2,..., of measurements 
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taken from samples corresponding to lots of products from a manufacturing process (e.g. number 
of defective items in each sample). Denoting by T the waiting time until the sampling level of the 
inspection changes using a /c-run switching rule associated with Z^s, we obtain the joint pgf of T 
and St {St denotes the total number of defective items observed until switching) by exploiting 
the fact that T follows a geometric distribution of order k. Finally, we demonstrate how the joint 
distribution of T and St can be useful in the estimation of the probability p of an item being 
defective, by employing an EM algorithm. 



2 Identities connecting the distributions of stopped sum and stop- 
ping time. 

Let Fi,i<2,... be a sequence of distributions on R such that J- R e wz dFi(z) < oo, i = 1,2,..., for 
every w in an interval W containing zero. We can always construct a sequence of independent rv's 
^1,^2,... on a probability space (Vl,F,F) such that Zj ~ Fi,i = 1,2,... . Moreover, if Fi{-\w) 
denotes the exponentially tilted Fi, i.e. Fi(x\w) := E(e wZl I^ i < 2 .])/E(e wZl ), w € R, we can always 
change P to a new probability measure F w on (ft, F) under which Z\, Z2, ... are still independent 
but now Zi ~ Fi(-\w), i = 1,2,... . A formal construction of the probability space (ft, F, F w ) is 
given in the Appendix. 

We shall write E w (-) for the expected value with respect to the measure F w . We shall also use 
the notation P := Po, E := Eo- It is easy to see that, in the special case when Z\, Z2, ... possess the 
same density / with respect to P, their density f w with respect to F w is given by 

e wz f(z) 
fw ^ = E(e«^)' 

Remark. (The derivative dF w /dP on F n ). Define F n = a(Zi, Z2, ■■■Z n ) C 7^, N to be the 
minimal a- algebra generated by Z\, Z2, ...Z n . The sequence T\, F2, ■■■ is a nondecreasing sequence of 
cr-algebras in 7£ N . The Radon-Nikodym derivative of P w with respect to P when both are restricted 
to F n is X n = e ™(Zi+z*+-+Zn) /Ytf =1 E(e wZi ) (that is, F W (A) = J A X n dF,Ae F n ) and hence 

¥ ( Y p w(Z 1 +Z 2 +...+Z n )\ 

for every T^-measurable random variable Y. It is worth mentioning that, even though P and F w 
are equivalent on every F n , they are mutually singular on Foo = 7£ N when w 7^ and Z±, Z2, ... 
are identically distributed (that is, there exist disjoint sets A, A' in 7?. N such that F W (A) = 1 and 
F(A') = 1). This can be easily seen since there exists a set B G B(R) such that F w (Zi 6 B) ^ 
F(Zi £ B) while (invoking the strong law of large numbers) ^ Y^i=i I{z t eB] converges to F w (Zi G B) 
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on some A £ 7£ N with P W (A) = 1 and to F(Zi G B) on some A' £ 7£ N with P(A') = 1. Since 
P tt (Zj eB) / P(Zj G 5) we have that 4n A' = 0. Thus P(A" n -)• 0) = 1 (see e.g. Theorem 35.8 in 
Billingsley (1986)) even though E(X„) = 1 for every n. Therefore, in general, there does not exist 
a Radon-Nikodym derivative of P^ with respect to P on 7£ N and hence P^, cannot be constructed 
on 7£ N from P through a Radon-Nikodym derivative. This fact does not induce any problem since 
we have guaranteed the existence of P„, via the Kolmogorov Existence Theorem (see Appendix) . 

Let now T be a stopping time associated with the sequence Z\,Z2, i.e. the set [T = n] = 
{w G : T(uj) = n} belongs to T n = u(Zi, Zz,...,Z n ) for every n = 1,2,..., and let St '■= 
Z\ + Z2 + ••• + Zt- The next result relates the distributions of T and St- 

Theorem 1 Let T be a stopping time associated with the sequence Z\,Zi-,-.- , and let Y be a 
random variable such that Y ■ I\r= n ] * s T n -measurable. Then 

T 

E(Ye wS ^I [T<oc] ) = i(r[jE(e^')/ [T<oo] ) (4) 

i=i 

for all real w such that the above expectations exist. 

Proof. If G k := Ye wS ? ££=1 1 [T=n] then \G k \ < \Y\ e^I [T<oo] a.s. and E(|Y|e^/ [T<oo] ) < 
00, which, by the Dominated Convergence Theorem (DCT), implies that E(limfc G k ) = lim k E(G k ). 
Thus, 

00 

E(Ye wST I [T<O0] ) = E( lim G k ) = lim E{G k ) = £ E(YL [T=n] e wS "). 

fe— >oo k — ^00 n=l 

By theorems' assumptions, the r.v. YI[T= n ] i s -^n-measurable and hence (see ^ above) E w (YI[ T = n ]) = 
E(YI [T=n] e wSn )/ Ui=i E(e wZi ). Therefore, 

00 n 00 T 

E(Ye^/ [T<oo] ) = £ E w (YL [T=n] U E(e wZ *)) = £ E w {YI [T=n] \\ E(e wZ >)) 

n=l i=l n=l i=l 

which, invoking again the DCT, leads to Q provided that E W (\Y\ l\J =1 E(e wZi )I [T<0O ]) < 00. ■ 

The above result can be considered as a version of Wald's Likelihood Ratio Identity (WLRI, 
see e.g. Siegmund (1985), or Lai (2004)). 

In the sequel we focus on a special use of Equation Q. Our aim is to generalize the following 
result of Antzoulakos and Boutsikas (2007): If Z\, Z2,.. is a sequence of iid binary rv's (trials) with 
P(Zj = 1) = 1 — ¥(Z{ = 0) = p and T denotes the waiting time (i.e. the number of trials) until a 
certain pattern £ occurs in Zi, Z2,.. then, the joint pgf of (T, St) follows from the pgf of T through 
the relation 

E(«V T ) =E W ((u(l -p + pw)f) (5) 
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for all u in sl neighborhood of 0, where the expectation is considered under ¥ w such that 
¥ w (Zi = 1) = 1 — ¥ w (Zi = 0) = pjj^__p ■ The above identity, reveals that, when the distribution of 
T is known then the joint distribution of (T, St) is also known. In other words, the distribution of 
T uniquely determines the joint distribution of (T, St) and consequently the distribution of St- 

A generalization of ([5]) could refer to any distribution for the Zj's and any stopping time T. 
In addition, an inverse form of ^ could also be very useful implying that the distribution of St 
uniquely determines the joint distribution of (T,St)- As it is shown in the next two corollaries, 
generalizations of this form can be easily derived from Equation Q . 

Corollary 2 If¥(T < oo) = F W (T < oo) = 1 then 

E{u T e wST ) = E w ({uE{e wZ )) T ), (6) 

for all real u,w such that the above expectations exist. In particular, E(e wSr ) = E w (E(e wZ ) T ). 

Proof. It follows from Q by letting Z\, Z2, - ■■ be a sequence of iid rv's and by setting Y = u T 
(note that u T I[T= n ] = ^ n ^[r=n] is -^n-measurable) • ■ 

Corollary 3 If there exists a real function w u such that E(e WuZ ) = u~ l and F Wu is a probability 
measure with F(T < 00) = ¥ Wu (T < 00) = 1, then 

E(u T e xST ) = E Wu (e (x - Wu)ST ), (7) 

for all real u,x such that the above expectations exist. In particular, E(u T ) = E Wu (e~ WuST ). 

Proof. By setting Y = u T e^ x - w ^ ST we have that the rv YI [T=n] = u n e (x-w u )(z 1+ ...+z n ) j^ = ^ 
is J-" n -measurable. Therefore by employing Q with respect to the measures P and F Wu we get 

E(u T e {x - Wu)ST e WuST ) = E Wu (u T e {x - Wu)ST E(e WuZ ) T ) 

which readily leads to Q since (uE(e WuZ )) T = 1. ■ 

It is worth mentioning that, more generally, we can similarly get from Q that, 

E(Yu T e wST I [T<oo] ) = E w (Y(uE(e wZ )) T I [T<oo] ) (8) 

and 

E(Yu T e xST I [T<oo] ) = E Wu (Ye^ x -^ ST I [T<oo] ), (9) 

where Y is a rv such that YI<T= n ] is -^-measurable. The above corollaries imply that, under 
appropriate conditions, the distribution of St uniquely determines the distribution of the stopping 
time T and vice versa. Two applications illustrating this fact are presented in the following section. 
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3 Applications 



3.1 The distribution of the first exit time of a random walk 

Let Zi,^2)— be a sequence of (non-degenerate) iid rv's representing the consecutive jumps of a 
random walk S n , n = 1,2,.. , that is, S n = Z\ + Z2 + ... + Z n . Define also the following stopping 
time 

T = inf{n : S n > b or S n < —a} 

for some a, b > 0. Obviously, T expresses the steps of the random walk until it exits the set (—a, b). 
It can be easily verified that E(T) < 00 (e.g. see Karlin and Taylor (1975), p. 264) and thus T is 
finite a.s. 

Probabilities regarding the first passage, or boundary crossing times arise in a variety of contexts 
in applied probability and statistics, such as sequential analysis, ruin theory, queueing theory, 
stochastic finance etc. Usually, it is of interest to evaluate the probability P(St > b) = 1 — P(St < 
—a), the distribution of T and E(T), V(T). 

In order to illustrate the applicability of identities ([6]) and Q we consider first the case when 
the jumps Zi are exponentially distributed (negative or positive with probabilities p and 1 — p 
respectively) and deduce explicit formulae for the pgf E(-u T ), the joint gf K(u T e xST ), the conditional 
pgf E (u t \St > b) and the expected values E(T) and K(T\St > b). We also consider the case a = 00 
corresponding to a random walk with only an upper barrier, which requires a different treatment 
(in this case T is not always a.s. finite). 

3.1.1 Random walk with exponentially distributed up and down steps 

(a) Denote by £ (9) the exponential distribution with parameter 9 > 0. For £ = 1,2, ... , let 



Zi 



Xi with probability p 
—Yi with probability 1 — p 



where X\,X2, ■■■ and Yi, Y2, ... are two sequences of iid rv's such that Xi ~ £(9\), Yi ~ 8(62)- It 
follows that the pdf / of each Zi is the mixture, f(x) = pfi(x) + (1 — p)/a(— x), where fi(x) = 
9ie~ 9iX , x > 0, and moment generating function (mgf) given by 

E(e wZ ) =p e wx h(x)dx + (1 - p) f°° e wx f 2 (-x)dx = + ilz^. 

J -00 J -00 0i -w 9 2 + w 

Initially, we find the probability P(Sr > b) via Wald's Identity by using a standard technique (see 
e.g. Karlin and Taylor (1975), p. 265). It can be verified that for w* = (1 —p)6\ —p02 we have that 
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E(e w ' z ) = 1, and therefore from g (or from Q) we get E(e w * ST ) = E w *(E(e w * z ) T ) = E W *(1 T ) 
1. Hence, it follows that 



1 = E(e™ bT ) = E 5t |S t > bj F{S T >b) + E [e w * T \S T < -a) (1 - ¥{S T > b)) 

and by solving with respect to F(St > b) we get 

l-E(e w * ST \S T < -a) 
P(5t ~ b '~ E(e™* s T\S T > b)-E(e w * s T\S T < -a)' (10) 
Invoking the memoryless property of the exponential distribution we have that 

E(e wST \S T >b) = e wh E(e w ^ T -^\S T -b^8(e l )) = -°^e w \ (11) 
v V / 9\ — w 

E (e wST \S T < -a) = e~ wa E ( e -<»(-«-«r)| — a — St ~ £(09)) = -^Ve"™, 

y V / W + 02 

and combining the above we deduce that for w* ^ 0, 

Pfc > aw ~ t»*+6> 2 (i-p)(e 1 +e 2 ) 

IT yoT _ w; g ie ^*b Q 2 e-™*°- 9 1 e(( 1 -P) e l-P e 2)b fl 2e -((i-p)9i-pfl 2 )° ' ^ ' 

01-10* w*+0 2 p(6»i+6> 2 ) (l-p)(6i+6 , 2) 

For w* = (i.e. the case where (1 — p)0i = p02) we can take w* — > in the above formula and 
subsequently deduce that F(S T > b) = eTq ^^g w 

Next, we derive the mgf of T by employing Corollary [3J A solution w u of the equation E(e wZ ) = 
u^ 1 with respect to w is 

_ e 1 -e 2 + tt ((i-p)0 2 -p0 1 )+ x /(0 1 -0 2 +^((i-p)e 2 -pe 1 ))2+4(i-t t )0 1 2 " n „-\ 

The function ui u is strictly decreasing for u G [0, 1] with wo = 9i,wi = max{0, (1 —p)9\ —^2) and 
thus < w u < 0i for u G (0, 1). Under the measure P w , the pdf / of each Zi takes on the form 

e™f(x) _ e wx (p9 ie -^I [x > 0] + (1 -p)9 2 e e **I [x<0] ) 



fw(x) 



E(e wZ ) ' ggi 4. (!-p) g 2 

9i—w O2+W 



J c w (^i - i^e-Pi-")*, x>0 

\ (l-c u ,)(^2 + ^)e-( 92+ ^(-^, x<0 

where c„ = ^(5^ + ^g?)" 1 (0 < c w < 1 for -0 2 < w < 0i). Hence, under P^u G (0, 1), 
we still have exponentially distributed up and down jumps, but now the parameters p, 9\ and 02 
are substituted by c Wu = g"^, , (9i — w u ), and {92 + w u ) respectively. Ag din, T is finite IP^-a.s. 
Using Corollary [3] and ( [TT| ) it follows that 

E( U T ) = E Wu {e~ w - ST ) 

= E Wu (e- w « ST \S T > b) F Wu (S T >b)+ E Wu ( e - w " ST \S T < -a) (1 - P Wu (S T > b)) 

- w z r > t) + £± w *r (i - *- (» a »»■ ("I 

(6*1 - w u ) + iy u (^2 + w«)-w u 
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Also, using (12) under the probability measure F w , we get 



1 9 2 +w u e -/3 u a 

I a l\ _ (l-c Wu )(6 1 +6 2 ) /-, c-n 



c^i+^r (l-c» u )(0i+e 2 )' 

where f3 u = (1- c Wu )(9i - w u ) - c Wu (0 2 + w u ). 



Combining (14) and (H5b we deduce the following proposition. 



Proposition 4 Let S n ,n = 1,2, ... be a random walk with step distribution F(x) = pF\(x) + (1 — 
p)F 2 (x), where Fi ~ 8(0i),i = 1,2, p £ (0,1). If T denotes the time until the random walk exits 
(—a,b),a,b > then the probability generating function ofT is given by 

( (01-viu) _ (9 2 +w u )e^ a \ (, _ (e 2 +w u ) 2 e-^ a \ 

, T _ \ e^ut e 2 ) ^ u(i-p)e 2 {e 1+ e 2 ) ) (gg + w u )e w » a 

u P 9i(0 1 +0 2 ) u{l-p)9 2 (9i+9 2 ) 



where 



P u = -VWi -0 2 + n((l - p)0 2 - P 9i)) 2 + 4(1 - u )0!0 2 , (16) 



w„ 



U0 1 -0 2 + u ((l-p)0 2 -p9 1 )-P u ). 



Note that, for the special case p = dl +Q 2 , the above generating function can also be derived by 
employing results established by Khan (2008). 

Apart from its theoretical interest, the above formula can also be used for the numerical deter- 
mination of the distribution of T for given values of the parameters 9±, 02, P, a and b, since 



1 d m 

m) = —j -y — (E(u T )) 
1 ml du mX v " 



(17) 



u=0 

In practice, this can be easily accomplished by the use of appropriate mathematical software (e.g. 
using the function SeriesCoef f icient of Wolfram Mathematica). In Figure 1 the distribution of 
T has been pictured for two sets of values of the parameters. The height of the bars represent the 
probabilities P(T = m), m = 0, 1, 50, while the small dots show the corresponding probabilities 
estimated by Monte Carlo simulation after 10 5 iterations. 

An explicit formula for E(T) can be easily derived by differentiating E(u T ) given in Proposition 
[4j with respect to u and taking u — > 1. The details are left to the reader. 

Employing Corollary [3j we can derive the joint gf of T and St, which yields 

E(u T e xST ) = E Wu (e xSr e~ WuSr ) 

= E Wu U x ~ w ^ St \S t > b) F Wu (S T > b)+E Wu U x ~ w ^ St \S t < -a) (1-F Wu (St > b)) 



w )Jx-w u )b (a — 1 ... \ (w u -x)a 



(01 - W u ) - (x - W u ) U (x - W u ) + (02 + W u ) 
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Figure 1: The probability mass function of T (p=l/3, 9±=2, 6*2=1, a = 8, 6 = 6 and 
p = 1/2, 1= l,6l 2 = 1, a = 4,6 = 4) 



where P^^Sr > 6) and io u are given above. 

Moreover, for the pgf of the conditional distribution of T, given that the random walk crossed 
the upper boundary, we observe that (l9j) with Y = L ST>b -i, x = 0, leads to 



IE {u T \S T >b)¥ (S T > b) = E(u T I [ST > b] ) = E Wu (e"^ J [Sr > 6] ) 



-w u St 



S T >b) F Wu (S T >b). 



Therefore we deduce the following result. 



Proposition 5 Let S n , n = 1,2, ... be a random walk with step distribution F(x) = pF\(x) + (1 — 
p)Fz(x), where F{ ~ £(8i),i = 1, 2. J/T denotes the time until the random walk exits (—a, b), a, b > 
i/ien i/ie conditional pgf of T given that St > b, is 



E (u T |5 T > b) 



n 



F(S T > b) 



-, uE (0,1) 



where w u ,P(St > 6) and¥ Wu (ST > b), are as in (16), (12) and (15) respectively. 



Proposition [5] along with (17) can be used for the calculation of the conditional probabilities 
h(m) = P (T = m\ST > b). In Figure 2, which was constructed similarly to Figure 1, we have 
plotted the conditional distribution of T for two sets of values of the parameters. 

Finally, it is worth mentioning that when the Z^s follow a Laplace distribution (i.e., 6\ = 62 = 
9,p = 1/2) the pgf of T takes on the simple form 

V ' l-u + (l + n)e( a+6 ) e " 
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Figure 2: The conditional probability mass function h of T, given that St > b. (p = 2/3, 9\= 1, 
9 2 = 2, a = 6, b = 8 and p = 1/2, 1= 1; 9 2 = 2, a = 5, b = 5) 



where u = \/I — u. Also, K(u T e xST ) now simplifies to 



E(u T e xST ) 



(1 



(1 + n 



+ 



CH-w) (9u-x)a 
x+8 e ' 



while the conditional pgf of T now reads 



E(u T \S T > b) 



e 6m {2 + a6 + W)(l - u){-u + e la6il {2 -u + 2u)) 



2adui 



(1 + a0)((l - e 2 ( a + b ) eu )(u - 2) + 2(1 + e 2(a+f>)e«) S - 

Finally, by differentiating K(u T ) and IE (u t \St > with respect to u, taking u — > 1 and after 
some algebraic manipulations, we may also easily derive explicit formulae for E(T), V{T) and 
E(n T |,ST > b). 

(b) We consider again the random walk Z%, Z 2 , ... discussed in (a) with a = oo (i.e. now there 
exists only an upper barrier), that is T denotes the waiting time (steps) until the random walk 
crosses b > 0. Exploiting the results of Section 2, we find the probability P(T < oo) and the 
conditional pgf of T given that T < oo. In this case, P(T < oo) = 1 only when the mean step 
K(Z) = j- — ^2 i s positive. We conveniently observe that the mean step under the probability 
measure F Wu is always positive, that is, 

l-c Wu p9iu (l-p)9 2 u 



w,. 



> o, 



for all u € (0, 1). This can be justified as follows: Note first that w u is strictly decreasing for u £ [0, 1] 
with wq = 9i and w\ = max{0, (1 —p)9\ —p9 2 }. It suffices to show that g(w u ) > 0, u G (0, 1), where 



g(x) = {9 2 + x) 2 p9 l 



x) 2 (l —p)9 2 . The function g(x) is strictly increasing in [0, 9\] {g'{x) > 



for x G [0, #i]). We examine the following three cases: 
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(i) Ifp0 2 -(l-i>)0i > 0, then^i = and hence g(w u ) > g[w{) = g(0) = (p0 2 -(l-p)0i)0i0 2 > 0. 

(ii) If p02 — (1 —p)0\ < 0, then w\ = (1 — p)9\ — p02 > and hence g(w u ) > g{w\) = p(l — p)(#2 + 
e^ 2 {{I - p)9 l - p6 2 ) >o. 

(iii) If p0 2 - (1 -pjflj = 0, then directly, E Wu (Z) = (jfe + > °" 



Therefore, FV,(T < oo) = 1,«6 (0, 1), and from relation (|9j) we deduce that 

E(n T / [r<oo] ) = E Wu (e- w ^I [T< ^)=E w Je- w ^\T<oc)P Wu (T<oc) 
= E Wu (e-^ s - \T<oo)= {91 ~ W ; )e ~ Wub , u € (0, 1). 

Letting u -> 1 we get that P(T < oo) = (glz^lgZ!^! , Since E(u T I [T<oo] ) = E{u T \T < oo)P(T < oo) 
we readily deduce the following proposition. 

Proposition 6 Let S n ,n = 1,2, ... be a random walk with step distribution F(x) = pF\{x) + (1 — 
p)F2(x), where F{ ~ £(6i),i = 1,2, p £ (0, 1). If T denotes the time until the random walk crosses 
b > 0, then the conditional pgf ofT given that T < oo is 

E(u T \T < oo) = ^I^V-i—^, u G (o, 1) 

V\ — W\ 



where w u is as in (16). Moreover, 

F T < co = 6l 

\ 1, (l-p)0i-^ 2 <O. 

By employing Proposition [6] we can easily compute the conditional probabilities s(m) = P(T 



m|T < oo) through (17). In Figure 3 the conditional probabilities s{m) have been plotted for two 
sets of values of the parameters. 

In the first case we have that P(T < oo) = e~ 3 ^ 10 ~ 0.69143, while in the second case 
P(T < oo) = 1. 



3.2 The distribution of the total number of defective items in a sampling system 
based on a A>run switching rule. 

In the current paragraph we present an application in acceptance sampling which is a major com- 
ponent of the field of statistical process control. In acceptance sampling we frequently deal with 
sampling systems/plans that have at least two sampling levels controlled by switching rules that 
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Figure 3: The conditional probability mass function s of T, given that T < oo (p = 0.4, 9\= 1.5, 
2 = 2, b = 3 and p = 1/2, 6 X = 1, 2 = 1, b = 3) 



are based on run and scan statistics. Two examples of such systems are the continuous sampling 
plans (see, for example, Schilling and Neubauer (2009)) and the Military Standard 105E (see, for 
example, Montgomery (2005)). 

In acceptance sampling for attributes we take samples of fixed size corresponding to consecutive 
lots of items from a manufacturing process and we record the number Zj, % = 1,2,... of non- 
conforming (defective) items in the i-th sample. Let c be the acceptance number of the "normal" 
sampling level, that is a lot is rejected if the corresponding sample contains more than c non- 
conforming items. Assume that a switch in a more "tightened" ("reduced") sampling level is 
instituted when each one of ^-consecutive samples have more than (less than or equal) c non- 
conforming items. We denote by T the waiting time (i.e. number of lots) until the sampling level 
of the inspection changes. Our aim is to obtain the joint pgf of T and St by exploiting the fact 
that T follows a known distribution. The study of the random variable St is crucial, especially 
under a rectifying inspection program. 

In the sequel we deal with a sampling system that begins under the normal sampling level and 
a switch is permitted only to the tightened one. More specifically, suppose that the size of the 
samples is fixed and equal to n and that the probability of an item being defective is equal to 
p G (0, 1). Therefore, each Zi, i = 1,2, ... follows a Binomial distribution with parameters n, p. The 
number T of inspected lots until the tightened sampling level is instituted can be expressed as 

T = inf{Z > k : Zi_ k+1 > c, Z { > c}. 

The stopped sum St = Y2lLi %i expresses the total number of defective items found until switching 
to the tightened sampling level. 

Since Zj's are discrete rv's we can conveniently set t = e w in Corollary [2] to get the following 
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relation for the joint pgf of (T, St), 

E(u T t ST ) =E t ((uE(t Zl )) T ), (18) 
where E(t Zl ) = (1 — p + pt) n . The distribution of the Z^s under the probability measure Ft is 

= *) = ^ ^ = (") (izfcf)'^)^ - = 0, 1, n. 

Therefore, under Ft, Z% follows a binomial distribution, with parameters n and 

Pt = ; T ; * > 0. (19) 

1 — p + pt 

The stopping time T can be considered as the first time a success run of length k occurs in a 
sequence of independent trials with success probability q = F(Zi > c). Hence, T < oo and the 
distribution of T is known as the geometric distribution of order k (see, for example, Philippou et 
al. (1983) or Balakrishnan and Koutras (2002)) with pgf given by, 

M(z, Q ) = E(z T ) = ^ _ { *f^J q $ zk+1 , ^[0,1]. (20) 
Under the probability measure Pt we have 



>u r t (Z l >c) = l-^2( n ) P ni-Pt) 

x=0 



and thus, Et(z T ), is given by (20), by replacing q with qt- Taking into account this observation, 



equality (18) leads to the following formula for the joint pgf of (T, St), 

E(u T t ST ) = E t ((u(l-p + pt) n ) T )=M(u(l-p + pt) n ,q t ) (21) 
(q t u(l - p + pt) n ) k (l - q t u(l -p + pt) n ) 
~ l-u(l -p + pt) n + (1 - q t )qt(u(l -p + pt) n ) k+l 

for all u S [0, 1] and t S (0, 1] guaranteeing that n(l — p + pt) n S [0, 1] and t > 0, as required by 



{ 20p and (|19J). 

The pgf E(t ST ) follows readily from the above by setting u = 1. The distribution of St, which has 
support {k(c+l), fc(c+l) + l, ...}, can be numerically evaluated for specific values of the parameters 



n,p,c and k as described after formula (17). Using this procedure we calculate F(St = m) for two 
sets of the parameters and the results are shown in Figure 4. 

It should also be mentioned, that since St is a positive integer- valued rv, the generating function 
W(t) = J2m=o^(^ T > m)t m ,t £ (—1, 1) of the tail probabilities can be easily determined via the 
formula 

E(t ST ) = 1 - (l-t)H(t). 
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Figure 4: The probability mass function of St 
n = 30,p = 0.2,c = 3,k = 3) 



(re = 20, p = 0.1, c=l, k = 2 and 



The tail probabilities of the distribution of St can be used in practice for the determination 
of the parameters of the above mentioned sampling plan. For various combinations of c and k 
it would be interesting to know the probability that the total number of defective items until 
switching exceeds a certain threshold. For example, consider the case where n = 40, c = 1, k = 3 



and p = 0.02. For u = 1, Equation (21 ) provides the pgf of St from which, by differentiation, we get 



that E(<St) = 142.04 (note that E(Sr) can also be evaluated via Wald's first equation). Moreover, 
using H(t), we can compute the percentile points of the distribution of St, which provide complete 
knowledge about the performance of the sampling plan, in terms of the total number of defective 
items found until the switching. Since in that case the median of the distribution of St is 100, 
we deduce that there is a probability lower than 50% that the total number of defective items will 
exceed 100 until switching. 

It is worth mentioning that the above procedure could easily be expressed in a more general 
setting. For example, if the measurements Zi, i = 1, 2, ... from the inspected lots follow a general 
distribution with cdf F (continuous, discrete or mixed) and a switching sampling level occurs at 
time T according to some stopping rule (e.g. a k/m scan rule), then following the methodology 
described above we can similarly determine the joint generating function of (T, St) provided that 
the pgf of T is known (e.g. is a geometric distribution of order k/m, see Balakrishnan and Koutras 
(2002)). In this respect we state without proof the following proposition. 

Proposition 7 Let Zi,i = 1,2,... be a sequence of iid measurements following a distribution F 
and let T be the waiting time (i.e. number of Zi's) until a switching sampling level occurs based 
on the k/m scan switching rule: k out of m consecutive Zi's belong to a specific measurable set 
A C R. If Mk ,m( z ,q) = ^{z T ),z G W denotes the pgf of the geometric distribution of order k/m 
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with success probability q, then 



for all u, w such that E(e wZ ) < oo and uE(e wZ ) € W. 

The interested reader who wishes to study the general sampling system which permits a switch 
from the normal sampling level to the tightened or to the reduced sampling level may consult 
Ebneshahrashoob and Sobel (1990) for the pgf of the associated waiting time rv T. 

3.2.1 Estimating p via an EM algorithm. 

In this last subsection we present an interesting application of the formula of K(u T t ST ) obtained 



above (cf. (21)), regarding the estimation of the probability p of an item being defective. Assume 
that v independent inspections are conducted according to the fc-run switching rule described above 
and let Tj be the waiting time (i.e. number of lots) until the sampling level of the i-th inspection 
changes, i = 1, 2, v. Denote also by St { the total number of defective items found until switching 
to the tightened sampling level has occurred in the i-th inspection, i = 1, 2, v. We are interested 
in estimating p when only the sample values r = (ti,T2, ■■■,t v ) of the v aforementioned waiting 
times are available. 

Since the likelihood function L(p; r) = n£=i ^C^i = T % I p) does not have a convenient form 
in order to directly find the MLE of p, we will show how we can alternatively employ an EM 
algorithm, considering S T = (S T1 , S T2 , S Tl/ ) as missing values (latent variables). The likelihood 
function L(p; r, S T ) now has the simple form 

V 

Up; t, S T ) oc TT p s ^ (1 - p) nT *- s -* = (_?_)Er=i s n (i _ p )»E£=i n m 

Since S r is not available, we can find the MLE of p by iteratively applying the following two steps 
(EM algorithm; cf. Dempster et al. (1977)): 

(E-step): Given r and the estimate of p at the j-th step, say p^\ compute the conditional 
expected value of the log likelihood function, 

Q(p\p®) = E ST|Tip(j) (logL(p;r,S r )) 

V V 

= V E(5 Ti \Ti = Ti,pW) log +uVr« log(l - p). 
The expected value E(5j , J?i = n) can be calculated by 



E(S Ti \Ti = n) = v (T l = Tl ) E mP (^ =m,Ti = t 
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which can be derived from Equation (21). More specifically, the sum ^2 m m¥(ST = m,T = r) is 
the coefficient of the r— th order term in the power series expansion of K(Stu t ) (where K(Stu t ) = 
^E(u T t ST )\t=i), and P(Tj = Ti) can be derived from the series expansion of E(it T ). 
(M-step): Find the parameter that maximizes Q(p \ p^), i.e., 

(j+i) mi i U)\ T!i=i^{ S T l \T i = T il p^) 
p VJ ^ ' = argmaxQ(p | p u >) = . 

v n Z^i=l T i 

The above two steps are repeated until we achieve the desired accuracy in the estimate p of p 
(e.g. p = pw'o) where jo = min{j : \p^ — p^~^\ < e). From the above procedure we can also get 
an estimate, E(firjTj = r«, p), of the unobserved variable S Ti ,i = 1,2, The observed Fisher 
information, which can be exploited for establishing approximate confidence intervals for p, takes 
on the form 



r /-N w ( a 2 logL(ff;r,S r 
J(p) = E St | Ti)5 — 2 



p=p / 



dp 2 

l —-- 2 I (1 - 2p) HS Tl \Ti = T i ,p) + np 2 ^T i ). 



(22) 



As an example of the above estimation procedure, suppose that a /c-run switching rule is em- 
ployed for v = 20 inspections with c = 4, = 3, n = 50, and the resulted waiting times are: 

r = (10, 5, 17, 4, 19, 3, 25, 6, 16, 16, 5, 4, 4, 5, 6, 12, 7, 12, 12, 13) 

(actually, these are simulated values with p = 0.10). By employing the EM algorithm we obtain 
p = 0.0998513 (e = 10 -8 ) while the estimates of S Tl ,i = 1, 2, ...,u are 

50, 27.4, 81.4, 22.4, 90.3, 19.4, 117.2, 32.4, 76.9, 76.9, 
27.4, 22.4, 22.4, 27.4, 32.4, 59, 36.4, 59, 59, 63.5 



The estimated standard error of p is I^p)- 1 ' 1 = 0.00299055 (cf. rt22h) and the approximate 
1 - a = 95% confidence interval for p is (p ± I{pY l/2 z a/2 ) = (0.0939898, 0.105713). 
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4 Appendix 

The formal construction of (Q,J-,F W ) : Denote by Q = R N the collection of all maps from 
N = {1,2,...} to R. Each element x of the product space R N can be written as a sequence 
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x = (x\,X2, •••) with each xi belonging to R. For each i G N consider the mapping Z± : R N — > R 
with Zj(x) = Xj (that is, is a coordinate function or projection). Let T = TZ N be the minimal 
cr-algebra such that Zi, Z2, ... are measurable, i.e. 7£ N := a(Z\, Z2, ■■■) = c({ x G R N : G B}, B G 
B(R), i G N), where £>(R) is the cr-algebra of the Borel sets of R. Next, denote by \i { the probability- 
measure on £>(R) that corresponds to F\,i = 1,2,... . For every i = 1,2,... define the distribution 
Fi(-\w) on R, such that 

Fi(x\w) := ( r °°' a| , r , x G R, w G W, 

which can be considered as the exponentially tilted Fi. Obviously, Fi(x\0) = Fi(x). If [i™ de- 
notes the probability measure on B(R) corresponding to Fi(-\w) then, equivalently, ^f(B) = 
J B e wx fj, i (dx)/ J* R e wx [i^dx) for every B G £>(R). Therefore fif « ^ i and the Radon-Nikodym 
derivative for fif with respect to [i i reads 

duV e wx 

— (-^-(a;) = -~ —r--, X G R, IW G W. 

Finally, invoking Kolmogorov's Existence Theorem, there exists a probability measure P w on 7£ N 
such that the coordinate variable process Zi,Z2, ••• on (R N ,7£ N ,]P W ) consists of independent rv's, 
with distributions (if,^,— respectively, and the construction is completed for all w G W. 
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